Online serving threshold and delivery policy adjustment

ABSTRACT

The present invention provides techniques for use in association with online advertising, relating to use of serving thresholds, associated with predicted click through rates, and delivery policies, associated with advertising inventory serving and distribution. An offline-trained machine learning-based model may be utilized in advertising serving decision-making in connection with serving opportunities, However, serving thresholds and delivery policies, for use in association with the model in serving decision-making, may he adjusted online, such as in real-time or near real-time, based on information obtained online affecting factors such as predicted click through rates and advertising inventory distribution.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/664,716, filed Jul. 31, 2017, which is a continuation of U.S.application Ser. No. 12/764,732, filed Apr. 21, 2010, the entiredisclosures of which are incorporated by reference.

BACKGROUND

In targeting, such as behavioral targeting, historical information suchas online user behavior information can be used in targetinghigh-performing advertisements to users. In this regard, a taxonomy,such as a hierarchical taxonomical tree of categories or topics, may beutilized, in which nodes in the taxonomy may represent categories ofinterest or behavioral targeting categories. A machine learning-basedmodel may be utilized in selecting advertisements for matching withserving opportunities and for serving to particular users. Historicaluser behavior information may he used to train the model offline. Forexample, such models may be trained periodically, such as monthly,weekly, or even more frequently, using updated historical user behaviorinformation.

Online, the offline-trained model may be utilized in advertisementtargeting and in determining or facilitating determination of suchthings as serving thresholds and delivery policies. Serving thresholdsmay include scores, which may directly correspond to particular CTRs,for instance. The serving thresholds may, for instance, relate tocertain minimum scores or CTRs that will be required for serving of anadvertisement in a particular category of the taxonomy, for instance (ofcourse, there are many details in the process which are not describedhere, for simplicity of explanation). Delivery policies may, forinstance, govern serving based at least in part on available advertisinginventory, or available inventory for a. particular taxonomicalcategory, etc. For example, serving thresholds may be set in eachcategory in such a way that both a desired level of performance,measured by category-specific CFR, and a desired volume of deliverablead impressions in the same category, are predicted to be achieved.Naturally, for a large-scale operation, scores and thresholds may be setbased on many other factors as well, and will take into account manyother variables across many advertisers, campaigns, etc. Generally, themodel may be used in making predictions and projections based on theoffline training.

Generally, serving thresholds and delivery policies are determinedoffline. Online, real-time or near real-time information, includingnewly obtained user behavior information, etc., can be fed into themodel, and the model can be used in determining when circumstances areright for serving, such as when a particular serving opportunity to aparticular user is predicted or projected by the model to meetrequirements such as the predetermined serving thresholds and deliverypolicy requirements.

As mentioned, such models are generally refreshed periodically byoffline training with newly collected historical user behaviorinformation. Models may only be refreshed so frequently, such asmonthly, weekly, or perhaps daily. However, circumstances, events, anddevelopments occur and change dynamically in real time, and the modelcannot be refreshed constantly to include such information as traininginformation. Such dynamic developments may include, as just one example,a breaking news event, which may affect anticipated CTRs or pertain tooptimal delivery policies, etc. Existing methods utilizing, for example,serving thresholds and delivery policies set using models that may havebeen refreshed offline may lead to suboptimal serving-relateddecision-making and determinations.

There is a need for methods and systems for improving or optimizingserving decision-making and determinations and associated servingthresholds and delivery policies.

SUMMARY

Some embodiments of the invention provide methods and systems for use inassociation with online advertising, relating to use of servingthresholds, associated with predicted click through rates, and deliverypolicies, associated with advertising inventory serving anddistribution. An offline-trained machine learning-based model may beutilized in advertising serving decision-making in connection withserving opportunities. However, serving thresholds and deliverypolicies, for use in association with the model in servingdecision-making, may be adjusted online, such as dynamically inreal-time or near real-time, based on information obtained onlineaffecting factors such as predicted click through rates and advertisinginventory distribution.

In some embodiments, for example, use of an offline-trained machinelearning- based model in combination with online-adjusted servingthresholds and delivery policies can provide for efficient and effectivetechniques for optimizing, or better optimizing, behavioral targetingand advertising inventory distribution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG, 1 is a distributed computer system according to one embodiment ofthe invention;

FIG. 2 is a flow diagram illustrating a method according to oneembodiment of the invention;

FIG. 3 is a flow diagram illustrating a method according to oneembodiment of the invention;

FIG. 4 is a flow diagram illustrating a method according to oneembodiment of the invention; and

FIG. 5 is a flow diagram illustrating a method according to oneembodiment of the invention.

While the invention is described with reference to the above drawings,the drawings are intended to be illustrative, and the inventioncontemplates other embodiments within the spirit of the invention,

DETAILED DESCRIPTION

FIG. 1 is a distributed computer system 100 according to one embodimentof the invention. The system 100 includes user computers 104, advertisercomputers 106 and server computers 108, all coupled or able to becoupled to the Internet 102. Although the Internet 102 is depicted, theinvention contemplates other embodiments in which the Internet is notincluded, as well as embodiments in which other networks are included inaddition to the Internet, including one more wireless networks, WANs,LANs, telephone, cell phone, or other data networks, etc. The inventionfurther contemplates embodiments in which user computers or othercomputers may be or include wireless, portable, or handheld devices suchas cell phones, PDAs, etc.

Each of the one or more computers 104, 106, 108 may be distributed, andcan include various hardware, software, applications, algorithms,programs and tools. Depicted computers may also include a hard drive,monitor, keyboard, pointing or selecting device, etc. The computers mayoperate using an operating system such as Windows by Microsoft, etc.Each computer may include a central processing unit (CPU), data storagedevice, and various amounts of memory including RAM and ROM, Depictedcomputers may also include various programming, applications, algorithmsand software to enable searching, search results, and advertising, suchas graphical or banner advertising as well as keyword searching andadvertising in a sponsored search context. Many types of advertisementsare contemplated, including textual advertisements, rich advertisements,video advertisements, etc.

As depicted, each of the server computers 108 includes one or more CPUs110 and a data storage device 112. The data storage device 112 includesa database 116 and a Serving Threshold and Delivery Policy AdjustmentProgram 114.

The Program 114 is intended to broadly include all programming,applications, algorithms, software and other and tools necessary toimplement or facilitate methods and systems according to embodiments ofthe invention. The elements of the Program 114 may exist on a singleserver computer or be distributed among multiple computers or devices.

FIG. 2 is a flow diagram illustrating a method 200 according to oneembodiment of the invention. At step 202, using one or more computers,during an offline period, a set of serving thresholds is initiallydetermined, to be utilized in online advertisement serving, in which aserving threshold is associated with a minimum anticipated click throughrate.

At step 204, using one or more computer computers, during an offlineperiod, a set of delivery policies is initially determined, in which adelivery policy is associated with one or more rules relating to servingof advertisements in accordance with required or optimal distribution ofadvertising inventory across serving opportunities.

At step 206, using one or more computers, during, and based at least inpart on information obtained during, an online period, adjustment isperformed of at least one of the set of serving thresholds to determineat least one adjusted serving threshold, and adjustment is performed ofat least one of the set of delivery policies to determine at least oneadjusted delivery policy.

At step 208, using one or more computers, during an online period, amachine learning-based model is utilized in decision-making with regardto serving of online advertisements in connection with servingopportunities based at least in part on the at least one adjustedserving threshold and the at least one adjusted delivery policy, inwhich the machine learning-based model is trained during an offlineperiod, and in which an online period is a period of activeadvertisement serving in which the model is utilized.

FIG. 3 is a flow diagram illustrating a method 300 according to oneembodiment of the invention, At step 302, using one or more computers,during an offline period, a set of serving thresholds are initiallydetermined, to be utilized in online advertisement serving, in which aserving threshold is associated with a minimum anticipated click throughrate.

At step 304, using one or more computer computers, during an offlineperiod, a set of delivery policies is initially determined, in which adelivery policy is associated with one or more rules relating to servingof advertisements in accordance with required or optimal distribution ofadvertising inventory across serving opportunities. The distribution ofadvertising inventory across serving opportunities relates todistribution of advertising inventory across serving opportunitiesassociated with different nodes of a hierarchical taxonomy of behavioraltargeting categorical nodes.

At step 306, using one or more computers, during, and based at least inpart on information obtained during, an online period, adjustment isperformed of at least one of the set of serving thresholds to determineat least one adjusted serving threshold, and adjustment is performed ofat least one of the set of delivery policies to determine at least oneadjusted delivery policy. The adjusted serving threshold and theadjusted delivery policy are adjusted at least in part to betteroptimize user targeting or advertising inventory distribution, based atleast in part on the circumstances occurring during the period.

At step 308, using one or more computers, during an online period, amachine learning-based model is utilized in decision-making with regardto serving of online advertisements in connection with servingopportunities based at least in part on the at least one adjustedserving threshold and the at least one adjusted delivery policy. Themachine learning-based model is trained during an offline period, usinginformation including historical user behavior information. Using one ormore computers, during an online period, the machine learning-basedmodel is utilized in decision-making with regard to serving of onlineadvertisements in connection with serving opportunities based at leastin part on the at least one adjusted serving threshold and the at leastone adjusted delivery policy. An online period is a period of activeadvertisement serving in which the model is utilized.

FIG. 4 is a flow diagram illustrating a method 400 according to oneembodiment of the invention. A machine learning-based model is depictedby block 404. The model 404 may be a featured-based model. Information,including advertising-related information, is used as traininginformation for the model 404. The information may be stored in a datastore, such as data store 410. The data store 410 can be or include anyof various forms or manners of data storage, and may or may not includeone or more databases. The information may include, for example,historical user behavior information, click through rate or otheradvertisement performance information, advertisement inventory anddistribution information, and other information. The information may becategory-specific, such as by being specific to nodes of a hierarchicalbehavioral targeting taxonomy.

The model 404 is trained offline periodically, as represented by block402. For example, the model may be trained monthly, weekly, daily, orotherwise.

Serving thresholds and delivery policies are initially determinedoffline, as represented by block 406, such as prior to an online period.An online period may be a period during which active onlineadvertisement serving takes place, including use of the model 404. Anoffline period may be period during which active advertisement servingusing the model is not taking place, or during which activeadvertisement serving associated with a particular active period is nottaking place, such as a period prior to an active serving period, orprior to a particular active serving period.

At step 412, online monitoring, tracking, and storing, such as in thedata store 410, is performed of category-specific advertisinginformation, including CTR and inventory distribution information,

At step 414, serving thresholds and delivery policies are dynamicallyadjusted online to optimize based on monitored advertising information.For example, the serving thresholds and delivery policies may beadjusted in real-time or near real-time, based on real- time or nearreal-time advertising information.

At step 416, online, the model, as well as adjusted serving resholds andadjusted delivery policies, are utilized in online advertising servingdeterminations/decision-making.

FIG. 5 is a flow diagram illustrating a method 500 illustrating oneembodiment of the invention. At step 502, offline training is performedof a machine learning-based model.

At step 504, offline setting of serving thresholds and delivery policiesis performed.

At step 506, dynamic, such as real-time or near real-time, monitoring isperformed of advertising-relevant information, such as category-specificadvertisement performance and inventory distribution information,assuming time remains in the pertinent online period.Advertising-relevant information can also include many other types ofinformation that may affect advertisement performance or optimaldelivery policies, such as time-based or news-based developments thatmay alter predicted click through rate for a particular behavioraltargeting category, for instance.

At step 508, the method 500 queries whether adjustments to servingthresholds or delivery policies are indicated based on the monitoredadvertising-relevant information. Various embodiments of the inventioncontemplate various ways by which particular adjustments may bedetermined, and by which underlying anticipated changes, such as changesin predicted click through rate for a particular category, for instance,may be determined or estimated.

If adjustments to serving thresholds and delivery policies are notcalled for based on presently available monitored information, then themethod 500 proceeds to step 512.

If adjustments are called for, then corresponding adjustments are madeat step 510, and then the method 500 proceeds to step 512.

At step 512, thresholds (including any currently adjusted thresholds)and the model 404 are utilized in targeting and serving determinations.

Following step 512, the method 500 return to 506, from which point newlyavailable information can be monitored and utilized, if time remains inthe pertinent online period.

Generally, some embodiments of the invention provide a framework thatcan be utilized to adjust serving thresholds and delivery policies, inonline advertising, such as to improve targeting performance orinventory distribution based on dynamically, real-time or near real-timemonitored category specific (such as behavioral targetingcategory-specific) click through rate (or other advertisementperformance) or inventory information.

As mentioned, some embodiments of the invention include use of anoffline- trained machine learning-based model, which makes use ofinformation including historical user information, in determinations anddecision-making regarding serving of particular advertisements toparticular users and in connection with particular servingopportunities. As a simple example, the model may be used online indynamically updating a score for a particular user based oncircumstances that may include activity of the user. The score maycorrespond with a predicted click through rate for the user inconnection with a particular behavioral targeting category, forinstance. If a user then appears online, for example, at a time within aparticular targeting time-frame, and the user's score at that time is ator above a particular threshold, then an advertisement corresponding tothat category may be served to the user in connection with an associatedserving opportunity. Serving thresholds and delivery policies may heutilized, for example, in achieving desired levels of advertisementperformance in connection with particular categories of a hierarchicaltaxonomy of user interest categories.

In some embodiments of the invention, thresholds and policies, initiallydetermined offline, are adjusted online in accordance with monitoredonline information, For instance, predicted click through rate may varywith many real-time or online-occurring circumstances, such as time oftime of day or week, or the occurrence of a breaking news or othersudden or real-time occurring or developing event, etc. The model,trained periodically offline, has not been refreshed to take intoaccount this information. However, adjustment of serving thresholds andserving policies can be utilized to account for and optimize based onthe monitored online information. The model can then be utilized alongwith the adjusted thresholds and serving policies to produce betteroptimized results. For example, the thresholds and policies can beadjusted in order to, for example, better achieve desired levels ofperformance in connection with particular categories, betterdistribution of advertising inventory across categories, etc.

For instance, a sudden event such as the death of an actor, or acelebrity scandal, can trigger a time-dependent spike in user interestin certain categories. As another example, shopping activity may bemonitored to peak during certain periods or holidays, or as a result ofcurrent events, such as prior to a predicted. rain storm or snow storm,for instance. Adjusted thresholds and policies can be used to betteroptimize, effectively and efficiently allowing taking into accountanticipated effects of such circumstances or events, which anticipatedeffects and the degree of such effects may be estimated or calculated inany of various ways.

In some embodiments, methods are utilized that include online monitoringof inventory or inventory distribution, and CTR, from the specifiedcategories, such as from or in a real-time or near real-time onlinebehavioral targeting scoring system (in which scores may corresponddirectly with predicted CTRs, for instance), and an advertisementserving system.

For instance, inventory information can be derived from or in part fromany of various metrics. These may include, for example, type and numberof events affecting a specified category during given time window,number of unique users affected in a specified category during a giventime window, number of affected users whose interest scores are abovecurrent category-specific serving threshold, for example, and variousother metrics and information. In some embodiments, a real-time or nearreal-time behavioral targeting system may obtain all advertising clickand view event information, allowing efficient calculation ofcategory-specific CTR in a given time frame, for instance. Once thecategory-specific CTR is obtained, it can be utilized in various waysand for various purposes, including calculating and determining servingthreshold adjustments, delivery policy adjustments, etc., in real-timeor near real-time.

In some embodiments, serving policies may be adjusted, for example, toprioritize serving in particular categories. As a simple example, aparticular user may be qualified to receive advertisements from a numberof categories, but available applicable serving opportunities to theuser may mean that only advertisements from some of the categories maybe served. In such circumstances, if monitored online information leadsto a determination that predicted CTR for a particular category iscurrently above that which was projected using the offline-trainedmodel, then that category can be prioritized in terms of advertisementserving, such as by adjusting a delivery policy to emphasize orprioritize advertisement serving in that category.

In some embodiments, for example, a ratio of the current predicted CTRfor a particular category to the projected CTR for the particularcategory using the offline-trained model, for a particular time window,can be used as at least one factor in determining delivery policy,where, for instance, a ratio greater than I may lead to a deliverypolicy adjustment that prioritizes that category for that time window.Furthermore, lower online- predicted CTRs can lead to adjustments thatde-emphasize associated categories, etc. Over time, by using thistechnique repeatedly over time, a desired inventory distribution can bemaintained while still increasing overall CTR, leading to increasedrevenue, etc. Generally, in some embodiments, online traffic patternsand pattern changes can be monitored and leveraged, through adjustmentsto serving thresholds and delivery policies, to better achieve ormaintain online advertising goals and metrics, for instance, in anefficient and cost- effective manner.

While the invention is described with reference to the above drawings,the drawings are intended. to be illustrative, and the inventioncontemplates other embodiments within the spirit of the invention.

1. A method comprising: during an offline period, determining a set ofserving thresholds for online advertisement serving; during an offlineperiod, determining a set of delivery policies for distribution ofadvertising inventory across advertisement serving opportunities; duringan online period, adjusting at least one of the set of servingthresholds to determine at least one adjusted serving threshold, andadjusting at least one of the set of delivery policies to determine atleast one adjusted delivery policy; and during an online period,modifying serving of online advertisements in response to advertisementserving opportunities based on a machine-learning model that is trainedon advertising information associated with serving advertisements duringan offline period and updated during an online period, and further basedat least in part on the at least one adjusted serving threshold and theat least one adjusted delivery policy, adjusted during the onlineperiod.
 2. The method of claim 1, wherein the advertising informationincludes user behavior information.
 3. The method of claim 1, whereinthe advertising information includes advertisement performanceinformation.
 4. The method of claim 1, wherein the advertisinginformation includes advertisement inventory information.
 5. The methodof claim 1, wherein the advertising information includes advertisementdistribution information.
 6. The method of claim 1, wherein theadvertising information is associated with a node of a hierarchicalbehavioral targeting taxonomy.
 7. The method of claim 1, furthercomprising periodically training the machine-learning model with theadvertising information.
 8. The method of claim 1, further comprisingadjusting at least one of the set of serving thresholds and at least oneof the set of delivery policies based on a news event occurring duringan online period.
 9. The method of claim 1, further comprising adjustingat least one of the set of serving thresholds and at least one of theset of delivery policies based on a chronological event occurring duringan online period.
 10. The method of claim 1, further comprising: duringan offline period, training the machine-learning model with theadvertising information; and during an online period, adjusting at leastone of the set of serving thresholds and at least one of the set ofdelivery policies based on a news event or a chronological eventoccurring during an online period and prior to re-training themachine-learning model during a next offline period based on advertisinginformation generated during the online period and associated with thenew event or with the chronological event.
 11. A system comprising oneor more computers and one or more storage devices on which are storedinstructions that are operable, when executed by the one or morecomputers, to cause the one or more computers to perform operationscomprising: during an offline period, determining a set of servingthresholds for online advertisement serving; during an offline period,determining a set of delivery policies for distribution of advertisinginventory across advertisement serving opportunities; during an onlineperiod, adjusting at least one of the set of serving thresholds todetermine at least one adjusted serving threshold, and adjusting atleast one of the set of delivery policies to determine at least oneadjusted delivery policy; and during an online period, modifying, basedon a machine-learning model that is trained on advertising informationassociated with serving advertisements during an offline period andupdated during an online period, serving of online advertisements inresponse to serving opportunities based at least in part on the at leastone adjusted serving threshold and the at least one adjusted deliverypolicy, adjusted during the online period.
 12. The system of claim 11,the operations further comprising: during an offline period, trainingthe machine-learning model with the advertising information; and duringan online period, adjusting at least one of the set of servingthresholds and at least one of the set of delivery policies based on anews event or a chronological event occurring during an online periodand prior to re-training the machine-learning model during a nextoffline period based on advertising information generated during theonline period and associated with the new event or with thechronological event.
 13. The system of claim 11, wherein the advertisinginformation includes user behavior information.
 14. The system of claim11, wherein the advertising information includes advertisementperformance information.
 15. The system of claim 11, wherein theadvertising information includes advertisement inventory information.16. The system of claim 11, wherein the advertising information includesadvertisement distribution information.
 17. The system of claim 11,wherein the advertising information is associated with a node of ahierarchical behavioral targeting taxonomy.
 18. The system of claim 11,the operations further comprising adjusting at least one of the set ofserving thresholds and at least one of the set of delivery policiesbased on a news event or a chronological event occurring during anonline period.
 19. A non-transitory computer readable medium or mediacontaining instructions for executing a method for use in associationwith an online advertising marketplace, the method comprising: during anoffline period, determining a set of serving thresholds for onlineadvertisement serving; during an offline period, determining a set ofdelivery policies for distribution of advertising inventory acrossadvertisement serving opportunities; during an online period, adjustingat least one of the set of serving thresholds to determine at least oneadjusted serving threshold, and adjusting at least one of the set ofdelivery policies to determine at least one adjusted delivery policy;and during an online period, modifying serving of online advertisementsin response to advertisement serving opportunities based on amachine-learning model that is trained on advertising informationassociated with serving advertisements during an offline period andupdated during an online period, and further based at least in part onthe at least one adjusted serving threshold and the at least oneadjusted delivery policy, adjusted during the online period.
 20. Thenon-transitory computer readable medium of claim 19, the method furthercomprising: during an offline period, training the machine-learningmodel with the advertising information; and during an online period,adjusting at least one of the set of serving thresholds and at least oneof the set of delivery policies based on a news event or a chronologicalevent occurring during an online period and prior to re-training themachine-learning model during a next offline period based on advertisinginformation generated during the online period and associated with thenew event or with the chronological event.
 21. The non-transitorycomputer readable medium of claim 19, the operations further comprisingadjusting at least one of the set of serving thresholds and at least oneof the set of delivery policies based on a news event or a chronologicalevent occurring during an online period.